The invention pertains to machine vision and, more particularly, to the inspection of electronic components and other objects. The invention has application in semiconductor device fabrication and electronic parts assembly, among others.
Machine vision refers to the automated analysis of an image to determine characteristics of objects and other features shown in the image. It is often employed in automated manufacturing lines, where images of components are analyzed to determine placement prior to assembly.
In the semiconductor and electronics industries, for example, machine vision is used to determine the position and orientation of semiconductor chips and other components before they are soldered into place on printed circuit boards. Typically, this is done by examining edges revealed in back-lit or silhouette images of the chips. This can be difficult, because the visual features (e.g., edge points) present in those images must be matched against internal models of the chips or other components. Nonetheless, it is necessary to permit accurate transformations from model to physical coordinates (i.e., the physical position of the chip). The process must be fast, moreover, so that automated equipment can perform up to speed. It must also be accurate enough, for example, to insure that chip leads make proper contact with boards or any other devices to which they are assembled.
The automated handling of semiconductor chips and other components further complicates machine vision processing of their images. Like other delicate electronic components, chips are typically manipulated during assembly by suction nozzles. These small vacuum tips are used to pick the chips out of bins, present them for inspection to the machine vision system, and place them on boards. Although the nozzles are usually arranged to pick up the chips from behind, they often protrude beyond the edges and, thereby, add extraneous features and complexity to the chip images. In some instances, corners or edges of the chips may be pulled unevenly into the nozzles, further complicating position analysis and subjecting the nozzles themselves to potential damage.
One technique for chip inspection is described in commonly assigned U.S. Pat. No. 5,974,169, entitled MACHINE VISION METHODS FOR DETERMINING CHARACTERISTICS OF AN OBJECT USING BOUNDARY POINTS AND BOUNDING REGIONS, filed Mar. 20, 1997. While this technique, as well as others by the Assignee hereof, have proven highly successful, there remains a need for still further methodologies suited to chip and other inspection applications.
An object of this invention is to provide improved methods and apparatus for machine vision.
A more particular object is to provide such methods and apparatus as permit the rapid, accurate and/or low-cost determination of object position and/or orientation.
A further object of the invention is to provide such methods and apparatus as facilitate the inspection of electronic and other parts during assembly or other automated operations.
Still further objects of the invention are to provide such methods and apparatus as require minimal computational resources.
Yet a still further object of the invention is to provide methods and apparatus as can be readily adapted to operate with existing digital data processing apparatus, regardless of whether designed for general purpose or special purpose machine vision applications.
The aforementioned objects are among those attained by the invention which provides, inter alia, machine vision methods for determining the location and orientation of objects in images.
Such methods, according to one aspect of invention, include performing multiple times an operation of matching a pattern against an image. The matching operation finds the location, if any, of a respective pattern in the image and determines the degree of match. The position and orientation of the object is determined from the results of one of the matching operations, for example, from the operation that revealed the highest degree of match.
Each pattern, in a method as described above, models one or more respective portions (e.g., edges) of the object. Each time the matching operation is performed (i.e., with respect to determining the location and orientation of a given object), a different pattern is applied. Though the patterns differ, they may model common portions of the object. Thus, for example, a first pattern may model a first side of the object; a second pattern may model a second, opposite side of the object; and, a third pattern may model both the first and second sides of the object.
The patterns are selected, according to a related aspect of the invention, so that at least one of them, individually, or two or more of them, in combination, matches (e.g., with an acceptably high degree of matching) at least a majorityxe2x80x94if not substantially allxe2x80x94of expected imaged portions of the object under expected conditions of execution of the method. Thus, for example, if the method is executed for purposes of determining the position and orientation of a semiconductor chip that is presented by a suction nozzle, the three exemplary patterns discussed in the preceding paragraph could be gainfully selected. This is so because, under normal circumstances and assuming proper pickup, at least one of those patterns individually (or two or more, in combination) will match substantially all portions of the chip that are not obscured by the nozzle in a backlit (e.g., silhouette) image and, therefore, will provide a basis for position/location determination. This is true regardless of whether the suction nozzle lands in the middle of the chip or toward either side.
Further aspects of the invention provide methods as discussed above where the position and/or orientation of the object are determined, not necessarily from the pattern that produced the highest degree of match in a matching operation, but from other factors in addition or instead. One such factor is a characteristic of the pattern being matched. Continuing the above example, a method employing the three aforementioned patterns (a first modeling a first side of the object, a second modeling a second side, and a third modeling both the first and second sides) may base a position/orientation determination on the third pattern so long as it achieves at least a minimal degree of matching, even if it produces a lower matching score than the other two patterns.
Yet still further aspects of the invention provide methods as described above adapted to determine the position and/or orientation of a square, rectangular or other generally rectilinear object, such asxe2x80x94by way of non-limiting examplexe2x80x94a semiconductor chip or other electronic circuit component. In these aspects, the patterns that are matched against the image are xe2x80x9cside regionsxe2x80x9d of the objects.
A side region, as used herein, is the border or edge on a side of the object and on nearby portions of one or two adjacent sides. A side region may includexe2x80x94in addition to or instead of nearby portions of one or two adjacent sidesxe2x80x94borders or edges of corners that define the transition to such adjacent sides. Thus, for example, the position and/or orientation of a square, rectangular or other rectilinear component can be determined using patterns that model, individually and together, two opposing side regions of the component.
Further aspects of the invention provide methods as described above including the step of inspecting the image to determine the condition of the object. According to these aspects, a position/orientation determination that is based on a match with a first of the patterns can be rejected in favor of a determination based on another of the patterns, e.g., if the inspection suggests that results from the first pattern are likely to be erroneous (notwithstanding the apparent degree of match between that pattern and the image).
Related aspects of the invention provide for rejecting a determination of position and/or orientation altogether, e.g., if inspection reveals that an imaged part is defective or, in the case of parts manipulated by suction nozzles or other handling apparatus, that the part is being held at such an angle as to preclude any reasonable determination of position/orientation. In the latter regard, methods according to the invention can include detecting missing or unexpected corners of the imaged object (either of which can reflect, for example, a miscuexe2x80x94such as a xe2x80x9ctombstonexe2x80x9d or xe2x80x9cbillboardxe2x80x9d pickup); detecting uneven or unexpected orientations of edges shown in the image (which, too, can reflect a miscue); and/or detecting unexpected edges internal to the imaged object.
Still further aspects of the invention provide methods as described above in which the component whose position and/or orientation to be determined is backlit for purposes of imaging. Related aspects provide methods in which the aforementioned patterns are generated by backlighting sample components. Alternatively, or in addition, those templates may be generated xe2x80x9csynthetically,xe2x80x9d e.g., from mathematical, algorithmic, symbolic or other descriptions of the components whose positions and/or orientations are to be determined.